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Abstract 

We prove a removal lemma for systems of linear equations over finite 
fields: let Xi, . . . , Xm be subsets of the finite field Fg and let A be a 
{k X ni) matrix with coefficients in and rank k\ if the linear system 
Ax — b has 0(5™^*^) solutions with Xi ^ Xi, then we can destroy all 
these solutions by deleting o{q) elements from each Xi. This extends 
a result of Green [Geometric and Functional Analysis 15(2) (2005), 
340-376] for a single linear equation in abelian groups to systems of 
linear equations. In particular, we also obtain an analogous result for 
systems of equations over integers, a result conjectured by Green. Our 
proof uses the colored version of the hypergraph Removal Lemma. 



1 Introduction 

In 2005, Green [31 Theorem 1.5] proved the so-called Removal Lemma for 
abelian groups. It roughly says that if a linear equation over an abelian 
group has not many solutions one can delete all the solutions by removing 



*Institute for Theoretical Computer Science (ITI), Faculty of Mathematics and Physics, 
Charles University, Malostranske namesti 25, 118 00 Prague, Czech Republic. E-mail: 
kral@kam.mff . cuni . cz. Institute for Theoretical computer science is supported as project 
1M0545 by Czech Ministry of Education. 

^Departament de Matematica Aplicada IV, Universitat Politecnica de Catalunya. E- 
mail: oserra@ma4.upc.edu. Supported by the Catalan Research Council under project 
2005SGR0258. 

^Departament de Matematica Aplicada IV, Universitat Politecnica de Catalunya. E- 
mail: lvena@ma4.upc.edu. Supported by the Spanish Research Council under project 
MTM2005-08990-C01-C02. 



1 



few elements. This Removal Lemma for groups has its roots in the well- 
known Triangle Removal Lemma of Ruzsa and Szemeredi (see also [7] for 
generalizations and many applications of this important result in combina- 
torics) which roughly says that if a certain graph has not many triangles, 
then they are supported over not many edges. 

In [5] , the authors gave a purely combinatorial proof, by using the Removal 
Lemma for graphs, of the algebraic version of the Removal Lemma for linear 
equations which allows for an extension of the result to non-abelian groups. 
In the same paper, the authors considered some extensions of the result to 
systems of equations in abelian groups which could be proved along the same 
lines. However to extend the result to general linear systems, the graph 
representation used in the mentioned paper presented serious limitations. 
Instead, the extensions to hypergraphs of the removal lemma, which have 
been recently proved by Nagle, Rodl, Schacht Gowers ^ or Tao jTTj . 
seem to be the natural tool to achieve this goal. 

Our main result is the following: 

Theorem 1 (Removal Lemma for systems of equations). Let F = '¥q he the 
finite field of order q. Let Xi, . . . , Xm he suhsets of F, A a {k x m) matrix 
with coefficients in F whose rank is k and b a k- dimensional vector over F. 

If there are o{q^~^) solutions of the system Ax = b with Xi G Xi, then there 
exist sets X[, . . . , X'^ with \Xi\X'- \ = o{q) such that there is no solution to 
the system Ax = h with Xi G X[. 

So, if a linear system has not many solutions, then these solutions are sup- 
ported by not many elements. Since this is the first time we use the little 
o notation, let us be more precise here: Theorem [T] asserts that for every 
e > 0, /c and m, there exists 5 > such that if the number of solutions is 
at most 6q"^~^ , all the solutions can be destroyed by removing at most eq 
elements from each of the sets X^. The value of 5 depends only on e, k and 
m, in particular, it is independent of q. 

Theorem [1] implies an analogous result in the framework where solutions 
from subsets Xi C [1,-/V] of a linear system in the integers are sought, a 
result conjectured by Green [3l Conjecture 9.4]. 

Independently of us. Conjecture 9.4 from [3] was proved by Shapira [10] 
whose method also yields a different proof of Theorem [TJ Shapira's proof 
also uses the colored version of the hypergraph Removal Lemma (Theorem[3]) 



2 



as our proof does. However, his proof involves 0(m^)-uniform hypergraphs 
and our proof involves {k + l)-uniform hypegraphs. At the high level, the 
two proofs follow the idea of reducing the problem to a hypergraph problem, 
but the particular ideas used to reduce the problem differ a lot. 

Let us also mention that the conclusion of Theorem [1] can be proven sub- 
stantially easier if we assume that every k columns of the matrix are linearly 
inpendent; we have reported on this result in [6j. 

As an example of application of Theorem [H we have the following. 

Corollary 2. // a subset X C Fg, q = , p > k > 3, has o{q^) arithmetic 
progressions of length k, then the set A has o{q) elements. 

The proof of Theorem [T] follows the main idea of the proof presented in [5] . 
As we have already mentioned, we use the edge-colored version of the hyper- 
graph Removal Lemma which follows from a more general result of Austin 
and Tao [H Theorem 2.1]. 

Theorem 3 (Austin and Tao [Ij). Let H be an edge-colored {k + l)-uniform 
hypergraph with m vertices. Let K = {V,E) be an edge colored {k + 1)- 
uniform t-partite hypergraph with M vertices. If the number of copies of 
H in K (preserving the colors of the edges) is o(M™), then there is a set 
E' C E of size \E'\ = o{M''+^) such that K' = {V,E\E'), as an edge-colored 
hypergraph, is H-free. 

2 Main result and its proof 

In this section, we present the proof of Theorem [1] except for an auxiliary 
lemma (Lemma H]) whose proof is given in the next section. We first explain 
the main steps of the proof. 

2.1 Outline of the proof 

We associate to the system Ax = b, where A has size kxm, an edge-colored 
{k -\- l)-uniform hypergraph H with m edges and m vertices. We shall 
construct a large m-partite {k + l)-uniform hypergraph K on mq vertices 
built up with m copies, Fi,F2, . . . , F„i, of the field F = ¥q. The edges of 
K are defined in such a way that each solution of the system corresponds 
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to q edge-disjoint copies of H with each edge representing an element of 
Xi. The bound on the number of solutions of our linear system translates 
to the fact that K contains o{q^) copies of H. By the Removal Lemma for 
hypergraphs, Theorem [3l we will find a set E' of edges with size o{q^^^), 
such that, by removing E' from K we delete all copies of H. We then apply 
a pigeonhole argument to reduce the o{q^^^) edges to o{q) elements from 
each set Xi using the fact that the q'^ copies corresponding to the same 
solution are edge-disjoint. 

The key point above is the construction of the auxiliary hypergraph. Before 
we explain the details of this construction, we show that we can assume the 
matrix A to be of a certain special form. 

2.2 Reductions of the system 

First observe that, by the nature of the statement of Theorem [H there is 
no loss of generality in assuming that the matrix A has full rank k. By 
permuting the columns and an appropriate choice of a basis of -F*", the 
matrix A can be assumed to be of the form A = {Ik\B) where Ik is the 
identity matrix. 

Our second observation is that it suffices to prove Theorem[T]for homogenous 
systems. Indeed, if A is written in the form [Ik\B), the general statement 
follows by applying it to the system Ax = once we replace the first k sets 
Xi,..., Xkhy X[ = Xi-bi,--- ,X'f^ = Xk-bk, where 6 = (61, ... , hm). 

We will make a further assumption on ^ = {Ik\B): any two rows in B 
have rank 2. Suppose on the contrary that rows Bi and Bj are not linearly 
independent, say Bi = XBj. This implies that every solution of the system 
Ax = satisfies Xi = X^^xj. Therefore we can replace Xi by X- = Xi n 
(A~^ • Xj^ , delete the j-th equation together with the j-th variable and apply 
our theorem in the resulting setting: the obtained system contains one less 
equation and one less variable. 

We may also assume that any row Bi has, at least, two non-zero entries. 
Otherwise the i'th equation would read Xi + bijxj = for some j G [k + 
l,...,?7i]. As in the preceding paragraph, we can replace the set Xj by 

Xj = Xj n (^—t>^j • Xi^ and consider the system obtained by eliminating the 
i-th equation and the i-th variable. 
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Consequently, we can assume that A and b satisfy the fohowing: 

(i) 6 = 0. 

(ii) The matrix A has the form A = {Ik\B) where is the identity matrix. 

(iii) Every two rows of B are linearly independent. 

(iv) Each row of B has at least two non-zero entries. 

Notice that the condition (iv) implies that ni > k + 2. 
2.3 Construction of the hypergraph K 

For the construction of the hypergraph K with the properties given in Sub- 
section 12.11 we shall use an auxiliary matrix associated to the matrix A 
which is described in Lemma HI Before we state the lemma, let us introduce 
some additional notation. If M is a matrix, the i-th row of a matrix M is 
denoted by Mj and denotes its j-th column. The support of a vector 
X £ F'^, denoted by s{x), is the set of coordinates with a nonzero entry. 

Lemma 4. Let A = {Ik\B) be a (m x k) -matrix with coefficients in ¥g. 
There are a (m x m) matrix C and m pairwise distinct {k + l)-subsets 
Si, ... , Sm C [l,m] with the following properties: 

1. AC = 

2. rank{C) = m — k (maximal subject to the first condition). 

3. For every i, s{Ci) C Si and s{Ci) ^ 0. 

4- For every i, there exists a subset S[ C Si with \S[\ = k such that the 
set of columns {C-', j € [1, . . . , m] \ 5-} has rank m — k. 

The proof of Lemma [H is postponed to Section [3l 

We are ready to define a suitable hypergraph representation of the linear 
system along the lines described in Subsection 12.11 Let Ax = be a linear 
system, where ^ is a A; x m matrix with entries in Fg satisfying the properties 
(i)-(iv) at the end of Subsection 12.21 Let C be the matrix associated to A 
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and Si,...,Sm be the {k + l)-subsets of [l,m] satisfying the properties 
stated in Lemma HI 

First, the hypergraph H is the (A; + l)-uniform edge-colored hypergraph with 
vertex set {1,2,..., m} and with edges Si, S2, ■ ■ ■ , Sm, where the edge Si is 
colored i. 

The hypergraph K is the {k + l)-uniform m-partite hypergraph with the 
vertex set F x [1, m]. For every x G Xj, K contains an edge {{aj,j),j G Si} 
if and only if 



and this edge is colored by i and labelled by x. Since the support s{Ci) is 
nonempty, K contains precisely edges colored by i and labelled by x for 
each x E and every color class of K form a simple hypergraph. 

We now show that the hypergraphs K and H have the properties given in 
the outline of the proof. 

Claim 1. If H' is a copy of H in K, then x = (xi, . . . solution of 

the system, where x^ is the label of the edge colored by i in H' . 

Proof. Let {(oi, 1), (02, 2), . . . , (am, be the vertex set of H' . By the 
construction of K, it holds that Ca = x where a = (oi, 02, . . . , am)- Hence, 
= ACa = Ax and x is a solution of the system. □ 

Claim 2. For any solution x = (xi, . . . , Xm) of the system Ax = with Xj G 
Xi, there are precisely edge-disjoint copies of the edge-colored hypergraph 
H in the hypergraph K . 

Proof. Fix a solution x = (xi, . . . , Xm) of Ax = with Xj G Xj, \ < i < m. 
First, we show that there is a copy of -ff in in which the edge colored i 
has label Xj, \ < i < m. 

Since the matrix C has rank m — k and satisfies AC = 0, the columns in C 
spans the solution space in and thus there is a vector u = (ui, . . . , Um) 
with X = Cu. In particular. 




m 
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where the second equahty follows from Lemma [J] ([3l). Therefore, for every i, 
the set {{uj,j),j G Si} is an edge of K colored i and labeled Xj. It follows 
that the edges {{uj,j),j G Si}, i = 1, . . . m, span a copy of H in K. Since 
the kernel of C is A:-dimensional, there are vectors u satisfying x = Cu, 
and each of them corresponds to a copy of H in K. We next verify that 
these copies are edge-disjoint. 

Let e = {{aj,j),j G Si} be an edge of K colored by i and labeled Xi G Xi. 
We show that all the copies of H in K contain different edges colored 
by i and labelled Xi. By Lemma d] dH), there is a subset 5- C Si of size k 
such that {C^ ,j ^ S^} is a set of m — /c linearly independent solutions of 
the system Ax = 0. Hence, we may find a vector u = {ui, . . . ,Um) with 
X = Cu such that Uj = aj for each j G S'^. With this choice, we must also 
have Uj = aj for each j G Si and the copy of H associated to this u contains 
the edge e. Hence, for each edge colored i and labeled Xi there is a copy of 
H associated to x in if which contains this edge. Since there are q^ such 
edges and the same number of copies of H associated to the solution x, no 
two copies can share the same edge colored i and labelled Xj. By applying 
the same argument to each of the colors 1, . . . , m, we conclude that the q^ 
copies of H associated to the solution x are edge-disjoint. □ 

We now proceed with the proof of Theorem [TJ 

Proof of Theorem\^ Since the number of solutions is o(g"^^'^), by Claims [2] 
and [H K contains o{q"^) copies of H. By the Removal Lemma for colored 
hypergraphs (Theorem [3]) , there is a set E' of edges of K with size o{q^'^^) 
such that, by deleting the edges in E' from K, the resulting hypergraph is 
H-iree. 

The sets X'- are constructed as follows: if E' contains at least q^ /m edges 
colored with i and labelled with Xi, remove Xi from Xi. In this way, the 
total number of elements removed from all the sets Xi together is at most 
m ■ o{q) = o{q). Hence, \Xi \X[\ = o{q) as desired. Assume that there is 
still a solution x = (xi,X2, • • • ,Xm) with Xi G X'-. Consider the q^ edge- 
disjoint copies oi H in K corresponding to x. Since each of these q^ copies 
contains at least one edge from the set E' and the copies are edge-disjoint, 
E' contains at least q^ /m edges with the same color i and the same label Xi 
for some i. However, such Xi should have been removed from Xi. □ 
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3 Proof of Lemma [4] 



In this section, we give an effective construction of the matrix C with the 
properties stated in Lemma HI 

We first define a sequence Bi, . . . , Bm of basis of the column space of A 
which is formed by columns of A. For a base Bi, Ti = {j G [1, m] : A^ & Bi} 
denotes the set of indexes of the columns of A contained in Bi. 

We set Tfc = [1, k], i.e., Bj. = {A^, . . . , A'^}. Since no row of the submatrix 
B is the zero vector, we may assume, up to reordering the columns from B, 
that Ai^k+i 7^ 0. 

Suppose that Bi has been defined for some k < i < m. Express each vector 
A^,i + l<j<mm the basis Bi, i.e., A^ = YlreTi ^rjA^. Let g{i + 1) be 
the smallest r such that brj ^ for some j S [i + 1,?ti]. By permuting the 
columns, if necessary, we can assume that ^ 0. The base Bi+i 

is then obtained from Bi by replacing the column with A^~^^, i.e., 

Ti+i = {Ti \ {g{i + 1)}) U{i + 1}. In particular, if i = A;, Tk+i = [2,k + 1] 
since / 0. 

We have now defined the bases B^, ■ ■ ■ ,Bm- Observe that 1 = g{k + 1) < 
• • • < g{m). Moreover, Tj C foi k < i < m. We further set Bq = Bm 
for our convenience. 

Suppose that Bi is defined for some < i < k. We proceed to define Bi-^-l 
in a similar manner. Let g{i + 1) be the smallest index in Tj \ such 
that the corresponding coefficient of the vector A^ expressed in the base Bi 
is non-zero. Note that g{i + 1) is well-defined since the vectors A^ , . . . , A^ 
are linearly independent. The base Bi+i is obtained from Bi by replacing 
with A'^'^^. In particular, it always holds {1, ■ ■ ■ ,i} <ZTi. Moreover, 
the base Bk defined in this way coincides with our original choice of it. 

An intuitive way of understanding the basis Bi and sets is as follows: the 
set Tj is the lexicographically maximal decreasing sequence of k indices from 
[i — m + l,i] (indices are taken modulo m) such that the columns with these 
indices are linearly independent. Let us prove this claim formally: 

Claim 1. The set Ti viewed as a subset of [i — m + and decreasingly 
ordered is the lexicographically maximal subset of [i — m + such that 
the columns A^ , j G Tj, generate the column space of A (indices are taken 
modulo m). 
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Proof. We prove the claim by induction for i = k, . . . ,m + k — 1. The claim 
holds if i = k as = {1, . . . , A;} and this is the lexicographically maximal 
subset of [k — m + 1, k]. Let T/ = {t[, . . . ,t'j^} be the lexicographically 
maximal subset of [i—m+l, i] such that j4*i , . . . , A** are linearly independent 
and let Tj = {ti, . . . ,tk}- For simplicity, assume ti > t2 > ■ ■ ■ > tk and 

t'l > t'2 > . . . > ffc. 

Let j be the first index such that tj ^ t'-. Clearly, t'- > tj. Since ti = t'l = i, 

it holds j > 1. Since j4*2, . . . , A*j are linearly independent, {tj, • • • , t'j-i} = 
{^2, • • • , ^ Ti-i \ {i2, • • • , and Tj-i is the lexicographically maxi- 
mal /c-subset of [i — m,i — 1] corresponding to linearly independent columns 
of A, Tj-i must contain an element r that is larger or equal to t'j. Clearly, 
g{i) = r (otherwise, r £ T^). 

If r = t'j, then the vectors A*i , A*2, . . . , A*^ are not linearly independent 
(as g{i) = r so that A'' is in the span of A*^, . . . If r > t'j, then 

the the sets of vectors A^^ , A*^ , . . . , and ^4*^ , . . . , , span the 

same linear space which implies that the vectors A^^ , . . . , A^^^'^ , N" , J^i 
are linearly independent. Consequently, Tj_i = {t2;---,^fc} U {r} is not 
lexicographically maximal independent /c-subset of [i — m, i — 1] (recall that 
t'2=t2, = tj-i and t'j >tj). □ 

The next claims will be needed to check that the matrix C which we define 
has the properties given in Lemma [H 

Claim 2. The function g : [l,m] — > [l,m] is bijective. 

Proof. In the described construction, i can only be inserted to Tj if j = i. 
If g{r) = g{s) = i for a pair of distinct r and s (which involves deleting i 
twice), then an element i would be deleted twice from Tj but inserted only 
once which is impossible. □ 

Claim 3. For every i = 1,2, ... ,m, the set Ti^i does not contain i. More- 
over, for every i = 2, . . . , k + 1 the set Ti_2 does not contain i. 

Proof. The claim holds by construction if i > k + 1. Hence, we have to focus 
on z G [1, k]. Since the columns A^ , j £ [1, m] \ {i} span the column space 
of A (by assumption (iv) on A), i ^ Tj_i by Claim [H 

To prove the second part of the statement, observe that, by assumption 
(iii) on A applied to rows i — l,i with 2 < i < k, the columns A^ , j G 
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[l,m] \ {i — l,i} span the column space of A. Again by Claim [H we also 
have i 

In the extremal case i = A; + 1 we also use assumption (iv) on A to ensure 
that there is j > A; + 1 such that A^, . . . ,A''~'^,A^ is a base. Again by 
Claim m /c + 1 0rfc_i. □ 



We can now define the matrix C. The j-th column of C has its support 
in Tj^i U {j}. For i G Tj^i, the entry Cij is the coefficient of A^ in the 
expression of A^ in the base Bj : 

A^ = a,A\ 

and Cjj = — 1 (recall that, by Claim [3l we have j Tj_i.) 

Clearly, each column of C belongs to the space of solutions of the system 
Ax = 0, so that Lemma U ([1]) holds. 

The submatrix of C formed by the last m — k columns and the last m — k 
rows is an upper triangular matrix with nonzero entries on the diagonal 
which implies that the rank of C is m — A;. This proves Lemma H] ^ 

We next define the family {Si, . . . , Sm} of {k + l)-subsets of [1, "m,]. By the 
definition of the function g and of the matrix C, the nonzero elements in 
the j-th column of C are in the rows U [g{j),'rn] if j € [l,k] and in 
the rows [g{j),j] if j € [k + l,m]. Let R C [l,m] x [l,m] be the set of 
subscripts defining this area, i.e, G if and only if either j G [1,A;] 

and i G [1, j] U [g{j),m] or j G [A; + l,m] and i G [gij),j] (see Figure[3]for a 
typical portrait of R.) 

When reading off this area in the matrix by rows we get the subsets Si, 
namely, 

= / U [hk], i G [1,A:] 

\ g'HT^-l)U{i}, ie[k + l,m]. 

By the definition of g, the support of the row Cj is contained in Si for every 
i G [l,m] and none of the rows is zero (the entry in the main diagonal is 
-!)• 

Let us observe that [S'il = A; + 1. It follows from the definition of g that 
g{i) ^ Ti-i and, for i G [1, A;], that the sets g^'^{[l,i]) and [i,k] are disjoint. 
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Figure 1: An example of the area R in matrix C which corresponds to the 
permutation g{l, 2, 3, 4, 5, 6, 7, 8) = (3, 4, 6, 7, 8, 1, 2, 5). 

Since g is a bijection (Claim [2|) the sets Si have cardinality A; + 1 for each 
i G [1, m]. 

Let us now show that the sets Si are pairwise distinct. Recall that the region 
R contains in column j G [1,^] the rows U [g{j),m]. It follows from 
the second part of Claim [3] that j ?j-2 for j = 2, k + 1, which implies 
g{j — 1) > j. Hence Sj does not contain j — 1 but it does contain j. On the 
other hand, the column j € [/c + l,m] contains in region R the rows [g{j),j], 
so again Sj contains j but does not contain j — 1- Let j < j' . If j' < k 
then {j' — C [j,k] C Sj, which implies Sj ^ Sj'. If j' > k then, either 
j' Sj or, as g is increasing in [A; + l,?7i], {j' — 1, j'} C Sj, which again 
implies Sj ^ Sj'. 

In order to prove the last part of Lemma HI we show that the columns 
{C^ , j ^ Si} form a set of m — A; — 1 linearly independent vectors. Together 
with Lemma H] ([3]) this fact implies Lemma H] (jl]) and completes the proof of 
the Lemma. 

Let C = {C^ : j ^ Si} he the submatrix of C formed by the columns 
with indices not in Si. We divide this matrix into four parts: the upper 
left UL = {Crs '■ r < i, s £ \ Si} formed by the first i — 1 rows of C 
and the columns with index at most i, the upper right UR = {Crs '■ < 
i, s G [i + l,m] \ Si} formed by the same rows and the remaining columns, 
the lower right LR = {Crs '■ r > i, s G \ Si} formed by the last 
m — i + 1 rows and the columns with index at most i and the lower left 
LR = {Crs '■ r > i, s (z [i + l,m] \ Si} with the remaining entries. 



11 



By our construction of the matrix C, UR is an all-zero matrix, while, as 
discussed in the proof of Lemma H] (l2|) , the columns with j E [i + 1 , m] \ 5'j 
are linearly independent. On the other hand, again by the construction of 
C, UL is an upper triangular matrix. It follows that the columns of C are 
linearly independent. The proof of Lemma 2] is now finished. 
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